The eects of system design alternatives on the acquisition of tax knowledge from a computerized tax decision aid

Accounting ®rms are intensifying their reliance on experiential learning, and experience increasingly involves the use of computerized decision aids [Messier, W. (1995) Research in and development of audit decision aids. In R. H. Ashton & A. H. Ashton, Judgment and decision making in accounting and auditing (pp. 207±230). New York: Cambridge University Press]. Accountants are expected to learn from automated decision aid use, because the aids are not always available when dealing with the aid's topical matter, and the knowledge inherent in the aid is needed for competency on broader issues. To facilitate knowledge acquisition and explain the logic of the underlying processes, computerized decision aids provide the rationale for their calculations in the form of online explanations. We study how the location of explanations in a computerized decision aid a€ects learning from its use. Speci®cally, this research extends the existing literature by using a framework for the study of learning from decision aid use and by using cognitive load theory to explain the failure of certain decision aid design alternatives to promote learning. We de®ne learning as the acquisition of problem-type schemata, and an experiment is performed in which cognitive load is manipulated by the placement of explanations in a computerized tax decision aid to determine its e€ect on schema acquisition. Schemata are general knowledge structures used for basic comprehension, and cognitive load refers to the burden placed on working memory when acquiring schemata. We ®nd that increased cognitive load produced by the location of explanations in a decision aid leads to reduced schema acquisition. Our results indicate that when explanations in a computerized decision aid are integrated into its problem solving steps, cognitive load is reduced and users acquire more knowledge from aid use. This appears to be an important design consideration for accounting ®rms buying or building computerized decision aids. # 2000 Elsevier Science Ltd. All rights reserved. This study investigates the determinants of knowledge acquisition from the use of an automated tax decision aid. Under the rationale of eciency and e€ectiveness in decision making, computer-based decision aids are commonly used in public accounting (Brown & Eining, 1997; Messier, 1995). However, assistance in making decisions is not the only expected function of these aids. It has been conjectured that experience garnered while using decision aids also promotes knowledge acquisition, because the aid should 0361-3682/00/$ see front matter # 2000 Elsevier Science Ltd. All rights reserved. PI I : S0361-3682(99 )00048-3 Accounting, Organizations and Society 25 (2000) 285±306 www.elsevier.com/locate/aos * Corresponding author. E-mail address: [email protected] (J.M. Rose), cjwolfe@ tamu.edu (C.J. Wolfe). provide an illustration of proper problem solving method, explanation of the method, and outcome feedback (Ashton & Willingham, 1988; Pei, Steinbart & Reneau, 1994). Pragmatically, experiential learning from the use of an automated decision aid is important for at least two reasons: (1) The aids will not always be conveniently available, and accounting practitioners often deal with client scenarios on an ad hoc basis; and (2) The base knowledge inherent in decision aids must be part of an accounting professional's repertoire, because as sta€ rise to managerial positions they must be able to evaluate the output of decision aids in a broader context. Knowledge has been shown to be a functional determinant of decision performance (Bonner & Lewis, 1990; Libby & Tan, 1994). Therefore, learning from using an automated decision aid is important to decision performance when making decisions inside an aid's domain without the use of the aid and when evaluating the ecacy of an aid's output. To understand the development of expertise in environments characterized by the use of automated decision aids, an important implication is that a detailed understanding of knowledge acquisition from using such aids is needed ®rst. Research on learning from computerized decision aid use has focused on two general questions: (1) How does experiential learning of computerized decision aid users di€er from hand calculation groups using traditional text-based materials; and (2) Can user or system attributes be manipulated to enhance learning from computerized decision aid use? Research results on learning di€erences between computerized decision aid users and hand calculation groups indicate that hand calculation treatments outperform aid users when given traditional text-based materials that facilitate a complete solution to the experimental problems (Glover, Prawitt & Spilker, 1997; Murphy, 1990). While these ®ndings are compelling, the bene®ts of decision consistency, eciency, and documentation apparently outweigh the sub-optimal learning experience of automated decision aid use, because accounting ®rms continue to make heavy use of such aids. Therefore, the more critical question is the second: Can anything be done to increase experiential learning when using computerized decision aids? Approaching this question from the side of the decision aid user, the earliest line of research addressed the possibility that mismatches between users' knowledge organization and the underlying structure of the decision aid led to learning de®cits (Frederick, 1991; Pei & Reneau, 1990; Ricchiute, 1992). These studies found that knowledge acquisition was improved when decision aid structures matched the knowledge structures of their users. However, any strategy based on these ®ndings shifts much of the training burden away from the experience of using the decision aid. Modifying the design of a decision aid to enhance its training capability is superior to training users on the knowledge structure of a decision aid, because complete experiential learning through automated decision aid use is more ecient. The design feature inherent in a computerized decision aid to assist learning is the aid's explanation facility (i.e. a software device that explains calculation logic). Early research comparing the presence or absence of explanations in a decision aid found explanations inconsequential to learning (Eining & Dorr, 1991; Murphy, 1990). Additionally, Steinbart and Accola (1994) found that more elaborate explanations did not promote a greater level of learning, and no learning e€ect was identi®ed for the di€ering placement of explanations within a decision aid (Mot, 1994; Odom & Dorr, 1995). The current study extends the existing literature by focusing on explanation placement within a 1 Demonstrating ®rm emphasis on experiential learning, Price Waterhouse Coopers has shifted a signi®cant component of their tax training to ``structured work assignments in the oce.'' Deloitte and Touche has also increased its emphasis on learning through experience. Employee manuals stress that in today's quickly changing ®nancial environment, on-the-job training ``is essential to maintain the level of competence necessary to render excellent service.'' 2 While Fedorowicz, Oz and Berger (1992) and Eining and Dorr (1991) found that computerized decision aid users learned more than hand calculation groups, equivalency di€erences existed in the decision support tools for the hand calculation and computerized treatments. 286 J.M. Rose, C.J. Wolfe / Accounting, Organizations and Society 25 (2000) 285±306